Skip to content
This repository was archived by the owner on Apr 14, 2022. It is now read-only.

Unpacking typing Tuples and Lists correctly #1553

Merged
merged 86 commits into from
Sep 16, 2019
Merged

Unpacking typing Tuples and Lists correctly #1553

merged 86 commits into from
Sep 16, 2019

Conversation

MikhailArkhipov
Copy link

#1498 BY @CTrando via db branch

Fixes #1457
Fixes #1494
Fixes #411
Fixes #1382
Fixes #1293

Mikhail Arkhipov and others added 30 commits June 20, 2019 09:16
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* Avoid empty node in empty global scope (#1226)

* Add null checks

* Avoid null node in empty global scope

* Remove unrelated change

* Lock content since it is locked in other cases (#1227)

* Fix addBrackets setting (#1241)

* Fix addBrackets

* Remove unrelated change

* Usings

* Handle library analysis better (#1246)

* Trying to augment argument set with Evaluator and Expression context (#1259)

* Moving anyStr to its own method to prevent FromTypeVar from being called with an ArgumentSet that has no context

* Trying to augment argument set with context wherever possible

* Setting expression in argument set, removing null

* Renaming to WithoutContext and updating comment

* Check for null args in PythonFunctionOverload

* When returning in init method, report a diagnostic error (#1261)

* When returning in init method, report a diagnostic error

* Adding check and tests for returning None before adding error for a return statement in the init function

* Cleaning up tests

* Give Diagnostic messages on improper usage of Generic (#1248)

* Adding diagnostic message for improper use Generic.

* Adding tests to make sure no diagnostics on valid uses of Generic

* Adding diagnostic error on binary operations with incompatible types (#1254)

* Adding diagnostic error for binary operations with incompatible types - e.g 5 + 'str'

* When NewType is called and the first argument is not a string, make a diagnostic message (#1260)

* Adding diagnostic error if the user calls NewType with the first arg not of string type

* Fix typo in TROUBLESHOOTING.md (#1285)

* Multiple analysis fixes (#1297)

Multiple analysis fixes.
- Fixes #1151: Reliable way to determine final analysis of a module
- Fixes #1201: NRE in SymbolCollector.AddProperty
- Fixes #1228: NRE in TryFindMissingDependencies
- Fixes #1294: Handle valid case for reloading AST
- Fixes #1295: FindReferences doesn't work on package init members
- Fixes #1296: DependencyResolver graph misses intermediate imports
- Part of the fix for #1174: All references are not listed when using ms language server (fixes in analysis itself are also required for this one)

* Rework search path resolution (#1289)

* first working path classfier, breaks existing path code for now

* inherit Comparer instead of implementing both

* don't do path depth comparison, preserve user ordering as before

* switch to using new path classification code in main module resolution

* fix typo

* clean up comment about paths in Server.cs

* use String.Split instead of regex when parsing script output

* test ordering of user provided paths

* add new normalize and trim helper, check preconditions as debug asserts rather than commenting on them

* use Split extension, don't MaybeEnumerate paths

* No diagnostic message when Generic is called with no args (#1305)

* No diagnostic message when Generic is called with no args

* When a class inherits from something that is not a class, give a diagnostic error (#1277)

* When analyzing class definitions, give diagnostic if bases are not all class types

* PR feedback
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Fix merge issue

* Remove debug code
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Basic locations test

* Navigation

* Add test

* Update baselines

* Port changes from dbtype
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Add RDT count to the analysis_complete event (#1396)

* Add RDT count to the analysis_complete event

* Use property instead of method

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Prevent crashes when the AST happens to be null (#1405)

* Remove all references to dropped modules and force GC on reload (#1402)

* First attempt at ensuring moduled dropped during reload are GC'd

* Make sure AST is set after content is cleared

* More tests passing

* ModuleWalker updates

* Also don't merge against specialized modules, do builtin _astMap reset before reanalyzing instead of after any analysis

* Don't call dispose directly, it's done elsewhere and has no effect

* Force GC via normal session instead of waiting for a hardcoded time

* Un-refactor function since it's not being called externally anymore

* Formatting/usings

* PR feedback

* Undo IsBuiltin, some things continued to be referenced with that changed

* Move reload logic down into PythonAnalyzer, remove added public inferface items

* Mode AddReference to PythonType as override

* Remove dead code, make all ResetAnalyzer calls full

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Update baselines
* Remove old qualified name

* Node storage

* Class and scope to use AST map

* Library analysis

* Fix SO

* Keep small AST with imports

* AST reduction

* Final field

* Initial

* Reload

* Ignore post-final requests

* Drop AST

* Remove local variables

* Test fixes

* Fix overload match

* Tests

* Add locks

* Remove local variables

* Drop file content to save memory

* Cache PEP hints

* Recreate AST

* Fix specialization

* Fix locations

* usings

* Test fixes

* Add options to keep data in memory

* Fix test

* Fix lambda parameters

* Fix argument set
Fix global scope node

* Fix overload doc

* Fix stub merge errors

* Fix async issues

* Undo some changes

* Fix test

* Fix race condition

* Partial

* Models and views

* Restore log null checks

* Fix merge conflict

* Fix merge issue

* Null check

* Partial

* Partial

* Partial

* Fix test

* Partial

* Partial

* First test

* Baseline comparison

* Builtins

* Partial

* Type fixes

* Fix type names, part I

* Qualified name

* Properly write variables

* Partial

* Construct module from model

* Test

* Variable creations

* Factories

* Factories

* Split construction

* Restore

* Save builtins

* Test passes

* Qualified name

* Better export detection

* Test fixes

* More consistent qualified names

* Sys test

* Demo

* Complete sys write/read

* Partial

* Partial

* Test staility

* Perf bug

* Baseline, remove debug code, deactivate db

* Test fixes

* Test fix

* Simplify a bit

* Baselines and use : separator

* Baselines

* PR feedback

* Merge master

* Remove registry reference

* PR feedback

* PR feedback

* Restore persistence + update test

* Better handle persistent module in dependencies

* Undo

* Add location converter abstraction

* Store member location

* Fix merge issue

* Basic locations test

* Navigation

* Add test

* Update baselines

* Type restore - initial

* Remove debug code

* Partial

* Fix stub merge

* Various model fixes

* Improve module handling

* Qualified name improvements

* Fix unbound case

* Improve stub merge

* Fix qualified names of typing

* Handle stub-only modules

* Add tests for io, re and sys

* Better handle named tuple

* Handle module circular references

* Handle else in platform and version clauses + handle it in symbol collector

* Formatting

* Fix walk of multi-level if statement

* Unify package search in imports

* Fix tests

* Undo change

* Port changes from dbtype

* Partial

* Named tuple support

* Baseline updates

* Debug code

* Support types

* Properly compare class member declaring type

* Nested classes and functions persistence

* Undo debug

* Fix numpy restore

* Baselines

* Fix tests

* Update AnyStr test reflecting changes to AnyStr behavior

* Exclude baselines from git diff

* Fix gitattr

* Move git setting to root

* Try no path

* Test fixes

* Undo change

* Additional stub merge fixes

* Baseline updates

* Fix goto def

* Protect specific type creation

* Track documentaton source

* More reliable tests + simplification

* Typo

* Cleanup

* Basic classification

* Fix merge error

* Module unique id fixes

* Stricted check to save analysis

* Revert "Fix tests"

This reverts commit 247a8c3.

* Revert "Unify package search in imports"

This reverts commit 67fed10.

* Don't clear scope variables with inner classes

* Fix typo

* Many new tests

* Fix collections test

* Fix CTypes

* Initial

* Update test

* Merge issues

* Fix CTypes again

* Fix null bases

* Tell between class members with/without self better

* TypeVar support

* Add bound/covariant

* Fix Enum reassignments

* Fix Random

* Fix import * over local declarations (Socket)

* Move interface

* Fix reference search

* Enable asyncio test

* More tests

* Enable dataclasses

* Add inspect and gzip

* More tests

* Add setting

* Add handling of import position relative to the variable

* Caching level changes

* Partial

* Rework

* Fix null

* Update baselines

* Functools pass

* Reverse stub merge

* Partial

* Partial ctypes

* Update variables post merge

* Fix ctypes

* Merge issues

* Fix typevar

* Add caching on construction

* Assorted fixes

* Named tuples

* Work around os stub oddness

* Fix generic qualified names

* Ignore lambdas

* Support named tuples as bases

* Baselines

* Test updates

* Named tuple tests

* PR feedback

* Using

* Move diagnostics to GenericBase

* PR feedback
…y! (#1470)

- Restore task handling in unit tests
- Throw exception from Debug.Fail instead of FailFast
MikhailArkhipov and others added 27 commits September 11, 2019 09:57
Fix import model restore
* Preliminary zip file

* Refactoring and adding .egg support

* refactoring

* More refactoring

* removing irrelevant classes

* Preliminary working in memory

* More things to make it work

* Getting rid of comment

* Refactoring

* More refactoring

* more changes

* refactoring

* Removing unneeded changes

* adding back

* PR feedback

* Test fix
* Handling unpacking with typing structures

* Removing changes to value enumerator

* Test fix, formatting

* Refactors

* Handling nested structures

* Handles non typing lists and tuples

* Giving up on fixing root problem, adding hack

* test fix

* Reverting change to test and fixing test

* Update syntax

* Checking #1494

* Handling #411 as well

* multiple assignment + unpacking fixes #1293

* Handling more cases with unpacking nested tuples

* merge fixes

* No default param for arg set, force people to use something
@MikhailArkhipov MikhailArkhipov merged commit 57a9a43 into master Sep 16, 2019
jakebailey pushed a commit to jakebailey/python-language-server that referenced this pull request Nov 1, 2019
Sign up for free to subscribe to this conversation on GitHub. Already have an account? Sign in.
Labels
None yet
Projects
None yet
4 participants